Bootstrapping Apprenticeship Learning

•We consider the problem of imitation learning where the examples, given by an expert, cover only a small part of a large state space. • Inverse Reinforcement Learning (IRL) provides an efficient tool for generalizing the partial demonstration, based on the assumption that the expert is maximizing an unknown utility function. • IRL consists in learning a reward function that explains the expert’s behavior, and that is a linear combination of state-action features. • Previous IRL algorithms use the empirical averages to estimate the expected feature counts under the expert’s policy. •We introduce a new technique for bootstrapping the examples. The proposed technique consists in iteratively learning a reward function. At each iteration, – the examples are replaced by a complete optimal policy given the current reward function, – the feature counts are calculated by using the current optimal policy and the dynamics model.